MulGT: Multi-Task Graph-Transformer with Task-Aware Knowledge Injection and Domain Knowledge-Driven Pooling for Whole Slide Image Analysis

Whole slide image (WSI) has been widely used to assist automated diagnosis under the deep learning fields. However, most previous works only discuss SINGLE task setting which is not aligned with real clinical setting, where pathologists often conduct multiple tasks simultaneously. Also, it commonly recognized that multi-task paradigm can improve efficiency by exploiting commonalities and differences across tasks. To this end, we present a novel framework (i.e., MulGT) for WSI analysis specially designed Graph-Transformer equipped Task-aware Knowledge Injection Domain Knowledge-driven Graph Pooling modules. Basically, Neural Network Transformer as building commons, our able learn task-agnostic low-level local information well task-specific high-level global representation. Considering different in depend on features properties, also design module transfer task-shared graph embedding into feature spaces more accurate representation Further, elaborately each both accuracy robustness of leveraging patterns We evaluated method two public datasets from TCGA projects, i.e., esophageal carcinoma kidney carcinoma. Experimental results show outperforms single-task counterparts state-of-theart methods tumor typing staging